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Abstract 


It  is  of  interest  to  understand  the  tradeoff  between  the  communication  resource  comsumption  and 
the  achievable  system  performance  in  networked  control  systems.  In  this  paper  we  explore  a  general 
framework  for  trade-off  analysis  and  decision  making  in  such  systems  by  studying  joint  quantization, 
estimation,  and  control  of  a  hidden  Markov  chain.  We  first  formulate  the  joint  quantization  and 
estimation  problem,  where  vector  quantization  with  variable-block  length  is  considered.  Dynamic 
programming  (DP)  is  used  to  find  the  optimal  quantization  scheme  that  minimizes  a  weighted  com¬ 
bination  of  the  estimation  error,  the  communication  cost,  and  the  delay  due  to  block  coding.  The  DP 
equation  is  solved  numerically  and  simulation  shows  that  this  approach  is  able  to  capture  the  tradeoffs 
among  competing  objectives  by  adjusting  the  cost  weights.  We  then  study  the  joint  quantization  and 
control  problem.  An  example  problem  is  solved  analytically,  which  provides  interesting  insight  into 
the  approach.  In  both  the  joint  quantization/estimation  problem  and  the  joint  quantization/control 
problem,  we  show  that  the  separation  principle  holds.  The  approaches  to  solving  these  two  problems 
share  the  same  spirit,  and  can  be  combined  and  extended  to  accomodate  more  objectives. 


1  Introduction 


Networked  control  systems  have  applications  or  potential  applications  in  defense,  transportation,  scien¬ 
tific  exploration,  and  industry,  with  examples  ranging  from  automated  highway  systems  to  unmanned 
aerial  vehicles  to  MEMS  sensor  and  actuator  networks.  Communication  in  networked  control  systems 
is  often  limited  due  to  the  large  number  of  subsystems  involved,  limited  battery  life  and  power,  and 
constraints  imposed  by  environmental  conditions.  Hence  an  important  concern  in  the  development  of 
networked  control  systems  is  how  to  deploy  and  allocate  the  communication  resources.  Proper  under¬ 
standing  of  the  tradeoff  between  the  communication  resource  consumption  and  the  system  performance 
will  help  to  make  such  decisions.  A  great  deal  of  effort  has  been  put  into  the  studies  of  control  systems 
when  communication  constraints  are  present.  In  particular,  stabilization  of  linear  systems  with  quantized 
state/output /input  has  received  much  attention  (see  e.g.,  [1,  2,  3]  and  the  references  therein).  Other 
problems,  such  as  the  state  estimation  problem  and  LQG  control  under  communication  constraints,  have 
also  been  studied  [4,  5]. 

*This  research  was  supported  by  the  Army  Research  Office  under  the  ODDR&E  MURI01  Program  Grant  No.  DAAD19- 
01-1-0465  to  the  Center  for  Networked  Communicating  Control  Systems  (through  Boston  University). 
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Figure  1:  The  setup  for  joint  quantization,  estimation,  and  control  of  an  HMM. 

In  this  paper  we  explore  a  general  framework  for  trade-off  analysis  and  decision  making  in  networked 
control  systems,  by  studying  jointly  optimal  quantization,  estimation,  and  control  of  a  Hidden  Markov 
chain.  Hidden  Markov  chains  form  an  important  family  of  Hidden  Markov  Models  (HMMs)  [6],  and  have 
been  widely  used  in  speech  processing,  computer  vision,  computational  biology,  telecommunications,  etc. 
Another  reason  for  us  to  choose  a  hidden  Markov  chain  is  that  numerical  and  even  analytical  solutions 
can  be  obtained  relatively  easily,  which  provide  insight  into  the  approach. 

Figure  1  illustrates  the  problem  setup.  Xn  is  the  state  of  a  homogeneous,  controlled,  hidden  Markov 
chain  taking  values  in  X  =  {x\,  •  •  •  ,  xs}  for  some  S  >  1.  The  output  Yn  takes  values  in  y  =  {y\,  •  •  •  ,  j/m} 
for  some  M  >  1.  Quantized  information  qn  of  the  output  is  sent  over  a  communication  channel  to  a 
remote  processor,  where  state  estimation  and  control  computation  are  performed  (later  on  we  shall 
justify  the  “separation”  of  estimation  from  control  shown  in  Figure  1).  The  control  Un  is  then  sent  back 
through  a  communication  channel  to  the  HMM.  To  highlight  the  main  ideas  and  simplify  the  analysis,  we 
assume  that  communication  channel  is  noise  free.  The  quantizer  is  allowed  to  be  time-varying;  however, 
the  variation  of  the  quantization  scheme  should  depend  only  on  the  information  available  to  the  receiver. 

The  control  Un  takes  values  in  U  =  {u\,  •  •  •  ,  uk}  for  some  K  >  1.  We  use  the  notation  Z\  to  denote 
the  sequence  of  random  variables  {Zi,  Zi+ 1,  •  •  •  ,  Zj}.  For  u  G  U,  1  <  i,  j  <  S,  we  let 

Ujij (u-)  —  / ^ !'oh  \ n  .  j  —  Xj | Xrj  —  x j ,  Un  —  //]  —  Prob[Xn+  \  —  x.j | Xn  —  x j ,  Un  —  u ,  Xq  ,  Uq  ]. 

We  assume  that  the  output  Yn  is  dependent  on  Xn  only.  For  1  <i  <  S,  1  <  j  <  M,  we  write 
CHj  =  Prob[Yn  =  Vj\Xn  =  Xi]  =  Prob[Yn  =  Vj\Xn  =  Xi,  X%~\  H0n,  yj1"1]. 

This  paper  is  divided  into  two  parts.  In  the  first  part  we  are  concerned  only  with  joint  quantization 
and  estimation,  i.e. ,  the  loop  in  Figure  1  is  not  closed.  Sequential  vector  quantization  of  Markov  sources 
was  considered  in  [7],  where  a  weighted  combination  of  the  entropy  rate  of  the  quantized  process  and 
the  compression  error  was  minimized.  We  note  that  such  a  “Lagrangian  distortion  measure”  appeared 
earlier  in  [8].  A  similar  approach  for  combined  classification  and  compression  was  proposed  in  [9].  We 
extend  the  work  in  [7]  to  the  case  of  vector  quantization  with  variable  block  length,  and  seek  the  optimal 
quantization  scheme  to  minimize  a  weighted  combination  of  the  estimation  error,  the  conditional  entropy 
of  the  quantized  output,  and  the  delay  due  to  block  coding.  The  problem  is  recast  as  a  stochastic  control 
problem  and  the  corresponding  value  function  satisfies  a  Dynamic  Programming  (DP)  equation  of  a  novel 
form.  We  further  investigate  numerically  solving  the  DP  equation  and  study  the  effects  of  weighting 
coefficients  on  optimal  quantization  schemes  through  simulation. 

In  the  second  part  of  the  paper  we  discuss  the  problem  of  joint  quantization  and  control.  Following 
the  same  spirit  as  in  joint  quantization  and  estimation,  we  seek  the  optimal  quantization  and  control 
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scheme  to  minimize  a  weighted  sum  of  the  communication  cost  and  the  cost  relating  to  the  system 
performance.  Here  we  restrict  ourselves  to  sequential  vector  quantization.  For  illustrative  purposes  a 
simple  example  problem  is  solved  analytically,  which  provides  interesting  insight  into  the  approach. 

In  both  the  joint  quantization/estimation  problem  and  the  joint  quantization/control  problem,  we 
show  that  the  separation  principle  [10]  holds.  Either  problem  is  decomposed  into  an  estimation  problem, 
and  a  decision  (quantization/control)  problem  based  on  the  state  estimation. 

The  structure  of  the  paper  is  as  follows.  In  Section  2  we  formulate  and  solve  the  joint  quantization 
and  estimation  problem.  The  resulting  DP  equation  is  numerically  solved  in  Section  3.  In  Section  4  we 
study  the  joint  quantization  and  control  problem.  We  finally  conclude  in  Section  5. 


2  Joint  Quantization  and  Estimation 

2.1  Problem  formulation 

We  consider  vector  quantization  with  variable  block  length.  Let  B  >  1  be  the  maximum  block  length. 
Given  the  initial  condition  for  Xq,  we  decide  the  first  data  block  Ij"1  with  n\  <  B  for  quantization  as 
well  as  the  actual  quantization  scheme  we  will  use  for  Y™1 .  Then  at  time  n\,  we  send  the  quantized 
y,ni ,  and  decide  the  next  data  block  Y^+1  (n\  +  1  <  ri2  <  n\  +  B)  and  the  associated  quantization 
scheme  based  only  on  the  information  available  to  the  receiver  (i.e. ,  the  initial  condition  of  Xq  and  the 
quantization  of  T”1).  This  process  goes  on  until  the  final  time  N  >  1  is  reached.  Time  instants  (e.g.,  0, 
n\  in  the  previous  discussion)  that  one  makes  decisions  are  called  decision  times.  We  assume  that  each 
transmission  is  completed  instantly  and  the  delay  due  to  communication  is  zero. 

We  now  formulate  the  problem  precisely.  Let  Hi  be  the  space  of  admissible  quantization  decisions  for 
Yf.  Here  by  a  quantization  decision ,  we  mean  a  scheme  for  both  division  of  Y^  into  (variable-length) 
blocks  and  quantization  of  these  blocks.  A  quantization  decision  is  admissible  if  at  each  decision  time, 
the  length  of  the  next  data  block  and  the  corresponding  quantization  scheme  are  decided  solely  based 
on  the  information  available  to  the  remote  processor  by  that  time.  This  makes  the  sender’s  decision 
transparent  to  the  receiver,  and  eliminates  the  need  to  transmit  the  quantization  scheme  separately.  On 
the  other  hand  this  imposes  the  requirement  of  certain  computation  capability  on  the  sender  side. 

Let  Ho  =  (7To(.Ti),  •  •  •  ,tto(xs))  be  the  a  priori  PMF  (probability  mass  function)  for  Xq,  where 
7To (xi)  =  Prob[X o  =  Xi],  1  <  i  <  S.  Given  Ho  and  u  G  Hi,  we  define  the  cost 

N 

J(n0,  w)  =  E[Y^  Jh(n )  +  A dJd(n)  +  \eJe(n)\.  (1) 

n=  1 

Here  Arf,Ae  >  0  are  weighting  coefficients,  and  Jh(n ),  Jd(n),  Je(n )  are  costs  relating  to  the  communi¬ 
cation  needs,  the  delay  due  to  block  coding,  and  the  estimation  error  at  time  n,  respectively.  To  be 
specific, 


Jh{n)  is  the  communication  cost  at  time  n.  In  this  section  we  assume  that  entropy  coding  [11] 
is  used,  so  the  (expected)  number  of  bits  required  to  transmit  a  random  vector  Z  is  bounded  by 
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H[Z]  +  1,  where  H[Z]  denotes  the  entropy  of  Z.  Hence 

h,  *  _  f  0,  if  no  transmission  at  time  n  ,  . 

\  H[Qn\Rn\  +  1,  otherwise  ’ 

where  Qn  is  the  information  transmitted  at  time  n,  Rn  represents  information  sent  before  time  n, 
and  denotes  the  conditional  entropy. 

•  Jd(n )  is  the  delay  cost  evaluated  at  time  n.  For  simplicity,  we  assume  that  Jd(n )  is  equal  to 
the  number  of  samples  being  delayed  at  time  n.  For  instance,  if  one  decided  to  quantize  Y^1 
as  a  block,  then  Jd(i  —  1)  =  1  (since  information  about  Yj_i  has  not  been  transmitted  at  time 
i  —  1),  Jd(i)  =  2  (since  information  about  both  Yi-\  and  Yt  has  not  been  transmitted  at  time  i ), 
Jd(i  +  1)  =  0  (no  backlog  at  time  i  +  1). 

•  Je(n )  is  the  estimation  error  for  Xn.  Assume  that  information  of  Yn  is  contained  in  the  quantized 

block  denoted  by  Qn+i2  for  h,*2  >  0.  Denote  the  information  sent  before  time  n  —  i\ 

by  Rn-h-  Let  tin  be  the  conditional  PMF  of  Xn  given  {no,  Rn-h,  Qn+i2},  and  let  tin  be  the 
conditional  PMF  of  Xn  given  {no,  Rn-h,  Then  we  define 

Je(n)  =  p(nn,nn), 

where  p(-,  •)  is  some  metric  on  the  space  of  probabilities  on  X .  In  this  paper,  we  take  p  to  be  the  l\ 
metric  on  M5.  Other  metrics  such  as  the  Kullback-Liebler  divergence  can  also  be  used.  Note  that 
for  ease  of  presentation  we  have  suppressed  the  dependence  of  IIn  and  Yn  on  appropriate  variables 
in  the  notation,  but  one  should  always  keep  such  dependence  in  mind. 


The  joint  quantization  and  estimation  problem  is  to  find  cu*  G  fh,  such  that 

J(no,u;*)  =  min  J(n0,u;)  :=  V(Uo). 

cjGUi 


(3) 


2.2  The  dynamic  programming  equation 


The  joint  quantization  and  estimation  problem  formulated  in  Subsection  2.1  can  be  recast  as  a  stochastic 
control  problem  and  be  solved  using  dynamic  programming.  As  we  shall  see,  the  conditional  PMF  Hn 
is  the  information  state  for  the  new  stochastic  control  problem  while  the  quantization  decision  u  is  the 
“control” . 


As  standard  in  dynamic  programming,  we  first  define  a  sequence  of  joint  quantization  and  estimation 
problems.  For  1  <  i  <  N,  we  let 

N 

Ji(ILi-i,Ui)  =  E[Y l  Jh{n)  +  A dJd{n)  +  A eJe(n)],  (4) 

n—i 


and 

Vj(n,_i)  =  min  (5) 

where  is  the  space  of  admissible  quantization  decisions  for  the  time  period  [i,  IV],  and  n,_i  is  the  a 
priori  PMF  for  A*_i,  i.e. ,  the  initial  condition  for  the  i-th  problem.  Clearly  for  i  =  1,  we  recover  the 
original  problem  formulated  in  the  previous  subsection. 
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We  denote  by  ©J  the  space  of  quantization  (encoding)  schemes  for  a  data  block  of  length  j,  say, 
Y^+j  1  for  k  >  1.  There  are  A'P  possible  outcomes  for  Y^+J~ 1 ,  so  each  Q]  £  ©J  partitions  these  M-7 
outcomes  into  groups  and  the  group  index  will  carry  (compressed)  information  about  Y^+]  1 .  In  this 
paper  we  are  concerned  with  estimation  of  Xn  and  not  reconstruction  of  Yn\  however,  considerations  of 
decoding  (to  the  space  y)  and  the  associated  compression  error  can  be  easily  accomodated  in  the  current 
framework  once  an  appropriate  metric  is  defined  on  the  discrete  set  y. 


A  recursive  formula  exists  for  n„.  Assume  that  for  i  >  1,  the  data  block  Y):8+Jl  1  of  length  j  is 
quantized  with  Q-7  £  Q-7  and  transmitted  at  time  i  +  j  —  1.  Let  Ri  represent  the  transmitted  information 
up  to  time  i  —  1,  and  IL;_i  =  (7Tj_i(.Ti),  •  •  •  ,  ib-i^s))  be  the  conditional  PMF  of  Xj_i  given  Ri.  Then  the 
estimate  of  A,*"1"-7-1  given  Ri  and  Q'7(T)l+J~1)  can  be  written  in  terms  of  Lh-i  and  QJ (Y-'+:)  1).  Indeed, 
by  the  Bayes  rule, 


Prob[X^j~l\Ru  Q>{Yi 


0(Vi+j-lW  = 


Prcb[X?i-'\R<\Pr6b[Q? (T/^'"1) | xX~\  Rt] 


Prob[Qi(Yi+j-l)\Ri] 
and  the  numerator  (the  denominator  is  just  a  normalizing  factor)  equals 
s 

Y.ProblX^-1^^  =  xa\ RilProbiQ^Y^-^xi^-^Ril 


s= 1 

S 


^ProbiXi.i  =  XslR^ProbiX^lXi.r  =  xs,  Ri]]Prob\Qi ,  R, 

S= 1 

5 

^  Yi-1(xs)Prob[xi+j-1\Xi_1  =  xs\Prob[Q^(Y^j-l)\X^j-\ 


S= 1 


by  the  Markovian  property. 


Therefore  we  can  write  the  conditional  (marginal)  PMFs  of  X‘+J  1  as 


/  n* 


V  n»+i-i 


(  f}(Ki-i,QHYi+j -1) 


\  Qj{Y^+j~l) 


for  some  functions  {/!  •■•  ,  /7 }  = :  fj.  We  now  work  out  fj  in  detail  for  j  =  1,2. 


(6) 


Example  2.1  For  j  =  1,  we  have 

MW  =  Prob[x,  =  xAqHy,),  n(_i] 

(Ef.i VgMan)  -  OWVMM 
iwteWUi  VSMfc)  =  CTO)*™)  ’ 

where  1  <  l  <  S,  and  l(-)  is  the  indicator  function. 


Example  2.2  For  j  =  2,  we  first  derive 

niti+i(xh,xi2)  =  Ero6[Ai  =  xh,Xi+l  =  xi2\Q2  (Yi+1)  Xi-i] 

_  (X^s=l  ’^i—i(xs)alisal2h)(Y2mi,m2=l  (l/mi j  Umi)  =  Q  Xi  ))clim1Q2m2) 

J^t1,t2=lX2s=l  7ri-l(xs)atisat2t1)(J2jn1,m2=l  1  (  Q2  (l/mi  j  2/m2  )  =  Q?Xi+  ))ct1m1Ct2m2) 
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where  1  <  l\,l2  <  S.  Then  we  have 


S 

(xh  )  ^  j  ('f'li  ?  ^12  )  ? 

(2  =  1 


S 

Hi+ 1(^/2)  ^  ^  ^~2,2+l  (^/i ;  ^/2  )  • 

«1  =  1 


(9) 

(10) 


Note  that  in  Examples  2.1  and  2.2  the  transition  probability  matrix  ( )  is  independent  u  since  the 
control  is  not  involved  here. 


Proposition  2.1  The  value  functions  {Vi\^=l  satisfy: 

Mlliv-i)  =  1  +  min,  H[Q\Yn)}  +  \eE\pUl(^N-i,  Q^ljv^rijv)], 
S^e1 

and  for  1  <  i  <  N  —  1, 


(11) 


VKH-i)  =  1  +  min  {J(J  01)Ad  +  min  {H[Q? (Y^1)] 

ie{l,2,-,min(B,Af-i+l)}  2  QJe0J  * 

i+j-l 

+\eE[  Y  p(fYi+1(^i-i, Q\rY~l))^n)  + ' , •./(// (i I-  I- Q^y/^-1)))]}}, 

n—i 

where  Yat+i(-)  =  0. 


(12) 


Sketch  of  proof.  For  i  =  N,  no  delay  is  possible  and  one  quantizes  Yv  only,  which  leads  to  (11). 

For  i  =  N  —  1,  one  has  the  choice  to  (a)  quantize  Y/v-i  alone  first  and  then  quantize  Yv  based  on 
quantized  Y/V-i ,  or  (b)  hold  on  until  N  and  quantize  Y$_ {  in  one  shot. 

1.  In  choice  (a),  no  delay  is  introduced  so  Jd(N  —  1)  =  0,  Jd(N )  =  0.  Let  w/v-i  =  (Qjy-n  2/v)  be 
an  admissible  quantization  decision,  where  Q1N_1,  QlN  £  B1  are  the  quantization  schemes  for  Yv_i  and 
Yv,  respectively.  Then 

wjv-i)  =  E{  1  +  H[Qn_i(Yn—i)]  +  Aep(r[jv— i  )  LItv— i ) 

+  l  +  H[Q1N(YN)\Q1N_1(YN_1)}  +  \ep(flN,flN)  ]  (13) 

N - - - 

Ti 

=  1  +  H[QlN  ^(Yn  - 1)]  +  F[Ae/3(rijv_i ,  fljv_i ) 

+  E[l  +  H[Q1N(YN)\Q1N_1(YN.1)]  +  \ep(flN,UN)\Q1N_1(YN_1)}  ].  (14) 

' - v - ' 

T2=JN(P.N-1,Q1N) 

Note  that  II a^— l  depends  on  Qjv-i^N-i);  an(l  Hjv  depends  on  both  Qjv-i(Yv_i)  and  Q}n(Yn).  Rewrit¬ 
ing  the  term  T\  in  (13)  as  T2  in  (14)  translates  the  requirement  that  Q] v  depend  only  on  Qjy-i^N-i) 
into  an  amenable  form.  In  particular,  the  optimal  Q}n  to  minimize  T2  will  depend  on  Q]v_1(YAr_i) 
through  IIat-i.  Minimizing  (14)  with  respect  to  ujn-i  and  plugging  (6)  for  IIat-i  ,  we  get 

Y^_1(nJV-2) 

=  1  +  min  {^[Q^Yjv-i)]  +  XeE[p(fl(UN_2,  Q^Yiv-i)),^-!)  +  Yv(/i  Obv-2,  Q^Yv-i)))]}. 
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2.  In  choice  (b),  we  have  Jd(N  —  1)  =  1  and  Jd(N)  =  0.  Minimizing  over  Q2  €  02,  we  have 
VLM jv-2) 

N 

=  1  +  Xd+  min  {!![<?(¥$_!)]  +  XeE[  ]T  p(^~N+2(UN.2,  fin)]}- 

Q2e02  n=N—\ 

Then  Fjv-i(IIiv-2)  =  min{V^_1(IIjv-2),  V^-_1(IIjv-2)},  which  satisfies  (12). 

Similar  arguments  can  be  used  to  prove  the  cases  for  i  <  N  —  2.  □ 

Remark  2.1  In  solving  (11)  and  (12)  one  also  obtains  the  optimal  quantization  policy  for  each  stage. 
Concatenating  the  optimal  quantization  schemes  (with  variable  block  length)  yields  the  optimal  quanti¬ 
zation  decision  for  our  original  problem  (3). 


3  Numerical  Results 


In  this  section  we  discuss  issues  in  numerically  solving  the  DP  equations  (11)  and  (12),  and  present 
simulation  results. 


3.1  Partition  enumeration 

In  solving  the  DP  equations,  we  need  to  enumerate  and  compare  all  partition  (encoding)  schemes  for 
the  finite,  discrete  sets  y3 ,  1  <  j  <  B,  where  y 3  is  the  product  space  of  j  copies  of  y.  Each  partition  for 
y 3  corresponds  to  an  element  of  0T  For  a  discrete  set  D,  the  number  of  partitions  grow  rapidly  with 
the  cardinality  no  of  D.  How  to  enumerate  all  partitions  without  repetition  is  an  important  issue  since 
repetitions  might  substantially  add  to  the  computational  complexity. 

We  have  developed  an  effective  method  to  eliminate  all  redundant  partitions.  The  procedure  consists 
of  two  steps.  In  the  first  step  a  tree-structured  algorithm  is  used  to  find  all  the  partition  patterns. 
Then  in  the  second  step  we  remove  remaining  redundant  partitions  by  making  use  of  the  “characteristic 
numbers” .  We  now  describe  these  two  steps  in  detail. 

By  the  partition  pattern  for  a  (disjoint)  partition  of  the  set  D  ,  we  mean  a  nonincreasing  sequence 
of  positive  integers  where  each  integer  corresponds  to  the  cardinality  of  one  cluster  in  the  partition.  For 
example,  if  we  partition  a  7-elenrent  set  into  3  clusters,  one  with  3  elements  and  the  other  two  with  2 
elements  each.  Then  the  partition  pattern  is  (3  2  2).  Different  partitions  may  share  the  same  partition 
pattern.  Our  enumeration  approach  is  to  first  list  all  partition  patterns,  then  list  all  corresponding 
partitions  for  each  pattern. 

To  list  all  patterns  we  construct  a  tree  as  follows.  Each  node  of  the  tree  has  two  properties,  “current 
value”  and  “remainder”.  The  root  node  has  “current  value”  =  0  and  “remainder”  =  n/>  The  root  has  njj 
children  whose  “current  value” s  are  nz >,  wd  —  1,  •  •  ■  ,  1,  respectively.  The  “remainder”  of  each  node  equals 
its  parent  node’s  “remainder”  minus  its  own  “current  value” .  For  a  non-root  node  with  “current  value” 
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0(5) 


Figure  2:  Tree-structured  pattern  generation  for  partitions. 


i\  and  “remainder”  i?.  it  will  have  iq  =  min(?'i,^2)  children  whose  “current  value”s  are  io,io  —  1,  ■  ■  •  ,  1, 
respectively.  A  node  is  a  leaf  if  its  “remainder”  is  0.  Every  path  from  the  root  to  a  leaf  is  identified  with 
a  partition  pattern  if  we  read  off  the  “current  values”  of  the  nodes  (except  the  root)  along  the  path. 

Figure  2  illustrates  the  pattern  generation  for  no  =  5.  For  each  node  the  number  inside  the  paren¬ 
thesis  is  its  “remainder”  while  the  number  outside  is  its  “current  value”.  We  immediately  have  all  the 
patterns:  (5),  (4  1),  (3  2),  (3  1  1),  (2  2  1),  (2  1  1  1),  (1  1  1  1  1).  We  note  that  if  the  “current  value”  of 
a  node  is  1,  then  either  it’s  a  leaf  or  it  has  only  one  child,  and  hence  there  is  only  one  path  passing  this 
node.  This  observation  helps  reducing  complexity  in  practice. 

Given  a  pattern,  we  generate  all  the  corresponding  partitions  by  choosing  appropriate  numbers  of 
elements  from  D  and  putting  them  into  groups.  Take  the  example  for  no  =  5  and  let  (3  2)  be  the 
pattern.  By  selecting  3  elements  for  the  first  group  and  leaving  the  rest  2  elements  for  the  second,  we 
can  list  all  10  partitions  for  this  pattern,  and  none  of  them  are  redundant. 

However,  if  an  integer  number  greater  than  1  occurs  more  than  once  in  a  pattern,  repetitive  enu¬ 
meration  of  certain  partitions  will  occur.  For  the  pattern  (2  2  1),  naive  enumeration  gives  20  partitions; 
however,  a  more  careful  investigation  reveals  that  only  15  of  them  are  distinct.  Such  redundancies  can 
be  virtually  removed  using  the  characteristic  numbers  of  partitions.  To  be  specific,  we  first  order  all 
elements  in  D  and  map  them  one-to-one  to  elements  of  {1,2,  •••  ,ud}  (denoting  the  map  as  X),  the 
latter  being  called  the  indices  of  the  former. 

Assume  that  the  current  pattern  of  interest  contains  two  ip  for  some  ip  >  1  (the  method  can  be 
extended  easily  to  the  case  of  more  than  two  ip  existing  in  the  pattern).  Given  a  partition  V\  consistent 
with  the  pattern,  we  locate  the  two  clusters  C\ .  C2  that  have  ip  elements.  We  then  let 

C!  =  E  W))3’  C2  =  E  W))3'  (15) 

deCi  deC2 

We  call  the  pair  (ci,  C2)  (after  being  ordered)  the  characteristic  numbers  of  the  partition  V\ .  By  modifying 
the  construction  method  of  characteristic  numbers  if  necessary  (although  the  one  in  (15)  works  quite  well 
for  the  many  cases  we  have  tested),  the  mapping  from  (61,62)  to  (ci,C2)  becomes  one-to-one,  and  we 
can  determine  easily  whether  the  partition  being  enumerated  has  been  listed  before  through  comparison 
of  the  characteristic  numbers. 
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In  addition  to  redundancy  removal  in  partitioning,  we  also  observe  that  in  (12)  many  evaluations 
repeat  for  each  i.  This  also  helps  to  speed  up  the  calculation. 


3.2  Simulation  results 

We  have  conducted  calculation  and  simulation  for  a  two-state,  two-output  hidden  Markov  chain.  The 
maximum  length  B  for  block  coding  is  2,  and  N  =  10.  The  matrices  ( atj )  and  (ctj)  we  use  are 

0.3  0.7  ' 

0.1  0.9  J  ’ 

By  varying  the  weighting  constants  A d  and  Ae,  we  compute  and  store  a  family  of  optimal  quantization 
policies.  Then  for  the  initial  condition  IIo  =  (0.9,  0.1),  we  obtain  50  sample  output  trajectories  by 
simulation.  Each  quantization  policy  is  applied  to  these  output  trajectories,  and  the  average  accumulative 
communication  cost  <7  ,  delay  Jd ,  and  estimation  error  Je  are  calculated. 

In  Figure  3(a),  each  curve  shows  the  variation  of  combined  communication  cost  and  delay  vs.  the 
estimation  error  as  Ae  is  changed  (A^  is  fixed  for  each  curve).  The  vertical  axis  is  Jh  +  \dJd  and  the 
horizontal  axis  is  Je.  We  have  also  found  that  (not  seen  in  the  figure),  for  A^  =  5.0,  the  accumulative 
delay  cost  =  0  (exclusively  sequential  quantization);  for  A^  =  0.9,  the  accumulative  delay  cost  =  5.0 
(exclusively  block-coding  of  length  2);  while  for  A^  =  1.15,  variable-length  block  coding  is  observed. 
Figure  3(b)  shows  the  variation  of  combined  communication  cost  and  estimation  error  vs.  the  delay  as 
A d  is  changed,  where  the  vertical  axis  is  Jh  +  A eJe  and  the  horizontal  axis  is  Jd.  From  the  figures  we 
see  that  jointly  optimal  quantization  decisions  vary  with  the  weighting  coefficients,  and  by  appropriately 
choosing  these  coefficients  we  can  achieve  the  desired  tradeoff  among  different  objectives. 


( aij )  — 


0.2 

0.8 


0.4 

0.6 


(a)  (b) 

Figure  3:  (a)  Weighted  combination  of  communication  cost  and  delay  vs.  estimation  error  (points  with 
lower  estimation  error  corresponding  to  higher  Ae);  (b)  Weighted  combination  of  communication  cost 
and  estimation  error  vs.  delay  (points  with  smaller  delay  corresponding  to  higher  A^). 
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4  Joint  Quantization  and  Control 


In  this  section  we  formulate  and  solve  the  joint  quantization  and  control  problem. 


4.1  Problem  formulation 

Consider  Figure  1.  We  now  restrict  ourselves  to  sequential  vector  quantization  of  Yn,  i.e.,  at  any  time 

n,  we  quantize  Yn  and  transmit  it.  Denote  the  quantization  scheme  at  n  as  Qn  E  01,  and  denote 

qn  =  Qn(Yn).  Let  5n  =  ( Qn,Un ).  We  call  {Sn}  jointly  admissible  if  Un  depends  only  on  {IIo,  ,  Uq-1}, 
and  Qn  depends  only  on  {IIo,  Uq-1}.  Fix  N  >  1.  We  denote  by  Ao  the  space  of  jointly  admissible 
quantization  schemes  and  controls  from  time  0  to  time  N  —  1. 

Given  the  initial  condition  IIo  f°r  Ao  and  {dn}^1  £  Ao,  we  define  the  cost  function 

JV— 1 

=  E[J2  A cJc(n)  +  Jp(n)},  (16) 

n= 0 

and  the  value  function 

m>)=  “in  J(n0,{<5n}^_1),  (17) 

{Sn  l^-^Ao 

where  Ac  >  0  is  a  weighting  constant,  and  Jc(n )  and  Jp(n )  are  the  costs  relating  to  communication  and 
performance  at  time  n,  respectively.  Jc(n )  takes  the  form  of  Jh{n )  in  Section  2  if  entropy  coding  for  qn 
is  used,  and  Jc(n)  =  log2 \ qn \  if  a  plain  coding  for  qn  is  used,  where  \qn\  denotes  the  number  of  possible 
outcomes  of  qn.  In  the  following  we  let  Jc(n)  =  h(qn)  for  some  suitable  function  h(-).  We  assume  that 
Jp(n)  depends  on  the  state  and  the  control,  Jp(n)  =  gn(Xn.  Un),  for  some  function  gn(-,  •). 


4.2  The  dynamic  programming  equation 

As  in  solving  the  joint  quantization  and  estimation  problem,  we  first  define  a  sequence  of  joint  quanti¬ 
zation  and  control  problems.  For  0  <  i  <  N  —  1,  we  let 

JV— 1 

JQUi,  {^If-1)  =  E[Y,  A ch(Qn(Yn))  +  gn{Xn,  Un)},  (18) 

n—i 

and 

Vlllii)  =  min  Jjn^l^jf-1),  (19) 

where  Aj  is  the  space  of  jointly  admissible  quantization  schemes  and  controls  from  time  i  to  time  N  —  1, 
and  Ilj  is  the  initial  condition  for  the  z-th  problem. 

We  denote  by  Ilj  =  {7tj(xi),  •  •  •  ,Tti(xs)}  the  conditional  PMF  of  A*  given  IIo,  <?o_1  (and  H16  corre¬ 
sponding  quantization  schemes),  and  Uq~1  .  We  can  derive  a  recursive  formula  for  flj, 

fij+i  =  /(flj,gj,£/j),  (20) 
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for  some  function  /(•,  •,  •).  To  be  specific,  for  1  <  l  <  S, 

-  /  \  _  Es=l  als(Uj):Xi(xs)('%2m=l  1(Qi{ym)  —  (li)cSm )  \ 

'+1  '  “  Ef=l^(z*)(E^=l  l(Qi(j/m)  =  ?i)ctm) 

Proposition  4.1  For  Q  G  01,  we  denote  by  AQ  the  space  of  functions  mapping  the  range  of  Q  to  U. 
The  value  functions  satisfy: 

Vjv-i(n  v-i)  =  min  min  E[Xch(qN-i)  +  E[gN_1(XN_1,  aN-i(qN-i))\qN-i\  ],  (22) 

Qjv-iS©1  CtN  —  i^A  N~1 

where  (Jn-i  =  Qjv-i(Ev-i),  and  for  0  <  %  <  N  —  2, 

^(Ilj)  =  min  min  E[Xch(qi)  +  E[gi(Xi,  «*(%))]%]  +  V^+i(/(IIi,  qh  «*(%)))],  (23) 

where  q%  =  Qt(Yt).  From  the  solutions  {(Q* , a*)}^1  to  (22)  and  (23)  one  can  construct  the  jointly 
optimal  quantization  and  control  schemes. 


Sketch  of  proof.  We  sketch  the  proof  for  the  cases  i  =  N  —  1  and  i  =  N  —  2,  and  the  rest  can  be  proved 
analogously. 

For  i  =  N  —  1,  we  have 

J(IIv-i,  (Qn-i,Un-i))  =  E[Xch(qN-i)  +  9n-i{Xn-i,Un-i)}  (24) 

=  E[\ch(qN-i)  +  E[gN- i(Xjv-i,  UN-i)\qN-i\  ]•  (25) 

Since  Un-i  is  measurable  with  respect  to  qN-i,  it  can  be  expressed  as  a^-iiqN-i)  for  some  a^-i  G 
A Qx-1.  Minimizing  over  Qn~i  and  a^-i  thus  gives  (22). 

For  i  =  N  —  2,  we  first  rewrite 

J{^-N-2,  =  E[Xch(qN-2)  +  E[g n -2(X N -2 ,  Un~2) \QN-2]  + 

E[\ch(qN-i)  +  gN-i(xN-i,  UN-i)\qN-2,  Un- 2]  ]•  (26) 

v  v  y 

Tz=Jn-i{IT^N-2,<1N-2,Un-2),{Qn-1,Un-i)) 

Since  Un-2  depends  only  on  qN-2i  we  have  Un-2  =  ajv-2 (qN-2)  fo1'  some  ajy-2  G  AQn~2.  Then  it’s 
not  hard  to  see  that  minimization  of  (26)  leads  to  (23).  □ 


Remark  4.1  By  rewriting  the  cost  functions  as  in  (25)  and  (26),  we  express  the  cost  in  terms  of  the 
“observables”  to  the  receiver.  As  in  the  joint  quantization/ estimation  problem,  the  conditional  PMF  TLn 
of  Xn  turns  out  to  be  the  information  state  (or  the  sufficient  statistic),  and  it  determines  the  optimal 
quantization/ control  scheme. 


4.3  An  example  problem 

We  take  the  machine  repair  problem  from  [12]  (pp.  190)  as  an  example.  A  machine  can  be  in  one  of 
two  states  denoted  by  P  ( Proper  state)  and  P  ( Improper  state).  If  the  machine  starts  in  P  and  runs 
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for  one  period,  its  new  state  will  remain  P  with  probability  | ,  and  if  it  starts  in  P.  it  will  remain  in 
P  with  probability  1.  At  the  beginning  of  each  time  period,  one  take  an  inspection  to  help  determine 
the  machine’s  state.  There  are  two  possible  inspection  outcomes  denoted  G  (Good)  and  B  (Bad).  If  the 
machine  is  in  P,  the  inspection  outcome  is  G  with  probability  if  the  machine  is  in  P,  the  inspection 
outcome  is  B  with  probability 

After  each  inspection,  one  of  two  possible  actions  can  be  taken,  C  (operate  the  machine  for  one 
period)  or  S  (stop  the  machine  and  perform  maintenance,  then  operate  the  machine  for  one  period). 
The  running  cost  for  one  period  is  2  units  if  the  machine  is  in  state  P.  and  is  0  if  it  is  in  P.  The  action 
S  makes  sure  that  the  machine  is  in  P  but  it  costs  1  unit. 

To  relate  this  problem  to  the  joint  quantization/control  problem  discussed  earlier,  we  assume  that 
the  inspection  outcome  needs  to  be  sent  to  a  remote  site  for  action  decision.  A  plain  coding  scheme 
requires  one  bit  to  send  the  information  G  or  B.  The  only  other  quantization  scheme  for  {G,B}  is  to 
cluster  these  two  outcomes,  in  which  case  no  information  is  sent  and  0  bit  is  required  for  communication. 
The  constant  Ac  now  carries  an  interpretation  of  communication  cost  per  bit.  A  different  interpretation 
of  Ac  would  be  the  cost  for  each  inspection  if  we  assume  that  the  inspection  is  optional. 

Given  an  initial  condition  of  the  machine,  the  problem  is  to  decide  at  the  beginning  of  each  time 
period  whether  to  communicate  the  inspection  outcome  and  what  action  to  take  based  on  the  received 
information,  so  that  the  total  cost  is  minimized.  One  can  show  that  the  value  function  of  this  problem 
is  concave  and  piecewise  linear.  We  have  obtained  the  explicit  solution  for  N  =  2.  Now  let  Prob[X o  = 
P]  =  |.  Then  one  of  the  following  four  joint  quantization/control  strategies  becomes  optimal  depending 
on  the  value  Ac: 

(a)  At  time  0,  send  the  inspection  outcome  Yq.  and  let  Uq  =  C(S ,  resp.)  if  To  =  G(B,  resp.);  At 
time  1,  send  Yi,  and  let  U\  =  C(S,  resp.)  if  Y\  =  G(B,  resp.); 

(b)  At  time  0,  send  Yo,  and  let  Uq  =  C(S,  resp.)  if  To  =  G(B,  resp.);  At  time  1,  if  To  =  G,  send 
Y\  and  let  U\  =  C(S,  resp.)  if  Y\  =  G(B,  resp.),  and  if  To  =  B,  let  U\  =  C  without  transmitting 
Ti; 

(c)  At  time  0,  send  To,  and  let  Uq  =  C(S,  resp.)  if  To  =  G(B,  resp.);  At  time  1,  let  U\  =  C 
without  transmitting  Yi; 

(d)  At  time  0,  let  Uq  =  C  without  transmitting  Yo;  at  time  1,  let  U\  =  S  without  transmitting  Yi. 

The  optimal  strategy  is 

(a)  if  Ac  <  ^ 

,  (b)  if  Ac 
(c)  if  i  <  Ac  <  § 

.  (d)  if  Ac  >  § 


In  Figure  4,  we  show  the  expected  accumulative  running  and  maintenance  cost  vs.  the  expected  bits 
of  communication  for  these  four  strategies.  The  thresholds  of  Ac  for  switching  of  the  optimal  strategy 
correspond  to  the  negative  slopes  of  the  line  segments  connecting  the  neighboring  points  in  Figure  4. 
Hence  when  the  communication  cost  per  bit  increases,  the  optimal  strategy  tends  not  to  transmit  the 
inspection  outcome. 
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Figure  4:  Running  and  maintenance  costs  vs.  communication  bits  for  jointly  optimal  strategies. 

5  Conclusions  and  Future  Work 

In  this  paper  we  have  studied  the  problem  of  joint  quantization,  estimation,  and  control  of  a  hidden 
Markov  chain.  We  first  investigated  the  joint  quantization  and  estimation  problem,  where  vector  quanti¬ 
zation  with  variable-block  length  was  considered.  Then  we  formulated  and  solved  the  joint  quantization 
and  control  problem.  The  common  theme  for  these  two  problems  is  that  a  weighted  combination  of 
different  costs  is  minimized.  By  varying  the  weighting  coefficients,  one  can  obtain  a  family  of  optimal 
quantization/control  schemes  that  reflect  different  tradeoff  strategies.  Simulation  and  a  simple  example 
have  been  used  to  illustrate  the  results. 

The  framework  presented  in  this  paper  can  be  extended  to  continuous-range  systems.  It  is  also 
possible  to  incorporate  the  case  of  noisy  communication.  Ongoing  work  involves  joint  consideration  of 
quantization  of  the  control  input. 
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